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ABSTRACT 



This position paper describes a concrete strategy to 
establish early warning systems or diagnostic instruments at the 
lower elementary school level in United Nations Relief and Works 
Agency (UNRWA) schools. The strategy described is designed to assess 
the quality of literacy and numeracy skills and to provide criteria 
for a realistic minimum threshold of schooling which would guarantee 
permanent retention of these skills. A conceptual framework for 
educational quality is provided, and a strategy for assessing quality 
is described for the following areas: literacy and numeracy skills; 
psychomotor skills and physical fitness in elementary schools; and 
the impact of organized violence in UNRWA school children in the 
Occupied Territories and Lebanon. The paper also briefly describes 
the instrument currently being developed at the School Education 
Division of UNRWA in Amman, Jordan. The paper concludes by 
highlighting the need for team effort, given the technical complexity 
of building diagnostic instruments and the lack of systematic 
experience in Arab^speaking countries. (MM) 
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from the original document. ''^ 





I 



POSITION PAPER : nTAf.NOSTIC FVAIIIATTOfJ 
OF NUMERACY AND I TTFR ACY SKTI I .S TM 
UNRWA SCHOOLS 



Prepared bv : 

DR. GONZALO RETAMAL 
CHIEF SCHOOL EDUCATION DIVISION 
UNRHA / HEADQUARTERS BRANCH 
AMMAN 



EXECUnVE SUMMARY 



This position paper takes as its main focus a concrete strategy 
that recommends to establish *early warning systems^ or diagnostic 
instruments at the lower elementary level in order to : 

(i) assess the quality of literacy and numeracy skills acquired by 
children in UNRWA schools; and 

(ii) to provide criteria for a realistic minimumi threshold of 
school i nq that will guarantee permanent retention of literacy 
and numeracy skills in children graduating or dropping-out 
from UNRWA schools (paras 2-3 ). 

Brief reference is made to the need for remedial action based on 

diagnostic testing (para 4). 

A conceptual framework for educational quality is provided and a 
diversified strategy for assessing quality as part of SED activities is 
briefly described for these main areas of test development. 

(a) Literacy and numeracy skills in L.E. education. 

(b) Psicomotor skills and physical fitness in elementary schools. 

(c) Impact of organized violence in UNRWA school children in the 
Occupied Territories and Lebanon, (paras 5 and 6). 

The paper concentrates also, on the first aspect of literacy 
skills by defining a conceptual basis for the elaboration of diagnostic 
testing, (para 7) and describes briefly the instrument at present being 
developed at SED which consists of : 

(a) a systematic sequence of skills for Mathematics and Arabic 
Language (or "map" of skills), 

(b) test modules, and 

(c) follow-up instruments, (para 8 and Fig 1). 
It is expected that this instrument will help to : 

(i) provide individualized identification of underachievers, and 

(ii) globally assess weaknesses in the teaching-learning process of 
basic and literacy skills (para 8). 



Finally, the paper highlights the need for team effort given 
the technical complexity of building diagnostic instruments and the lack 
of systematic experience in the Arab speaking countries. Outside 
expertise and collaboration with specialized sister agencies, NGO's 
among others is highly recommended (paras 9 & 10). 
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Diagnostic Evaluation for Literacy and Numeracy Skills 

Tn unrwa schools 



1. Two basic documents have been prepared by CSED in order to 
delineate a strategy vis-a-vis the problem of literacy and numeracy in 
the UNRWA schools (1). These papers were presented to the consideration 
of the Director of Education during the FEDO's Conference in Vienna in 
October 1991. In these papers 2 main recommendation were put forward so 
as to : 

2. Establish "early warning system *' or in other words, to 
develop and introduce diagnostic tests at the lower elementary level in 
order to monitor the quality of literacy and numeracy skills acquired by 
children during the first grade of primary education. 

By developing these instruments SED expects to establish a 
rather standarized and objective measure of educational quality based on 
the level of maturity and development of the child in the L.E. cycle, 
rather than to link evaluation only to a normative criteria determined by 
the official syllabus. 



3. The second aspect concerning diagnostic testing on the basic 

literacy and numeracy skills is related to the problem of retention of 
basic skills in children that leave UNRWA schools at an early stage via 
drop-out and/or graduation and incorporate to productive activities. 



(1) CSED/ Li teracy in UNRWA schools : A preliminary review of evidence'. 
(Paper presented to the FEDO's Conference, UNRWA Department 
of Education, Vienna, October 1991 ) internal document. 
CSED, 'Working Paper : Development of diagnostic testing and 

remedial structures at the Lower Primary Level in the Field'. 
(Paper presented to the FEDO's Conference, UNRWA Department of 
Education, Vienna, October 1991) internal document. 



On this aspect the problem of providing a minimum threshold of 
school i nq which is required to guarantee permanent retention of literacy 
and numeracy skills needs to be addressed and monitored. (2) Diagnostic 
instruments on 1 i teracy and numeracy, adapted and standardized for 
children ages 6 to 10 will help SED to define a more realistic 
assessment according to the diversity of UNRWA schools. In other words, 
an absolute norm (i.e end of 4th grade) for a minimum threshold of 
education and literacy, applied across the board to all UNRWA schools 
will not be possible or realistic, unless all the factors involved in 
literacy retention, such as individual or collective need, social and 
political stabi li ty affecting schools, availability or reading 
materials, type of instruction provided, language being learned, or the 
quality of teaching, etc could be held constant (3). 



This aspect seems relevant at the light of preliminary research 
in West Bank and the results of Brevet examinations in Lebanon, showing 
that at the end of the basic education cycle, an important number of 
students from UNRWA schools have not acquired functional numeracy and 
literacy skills. 



Remedial measures 



4. The second consideration in the strategy defined by SED was to 

establish on the basis of diagnostic ev:.luation some effective remedial 
action. Two specific proposals have been put forward in Lebanon and 
Syria as pilot experiences on the basis of which improvement of literacy 
and numeracy and basic skills can be insured. 



(2) Hamada'-^he, Ali et al . , Theory and Practice of Literacy Work : 
Policies> strategies and examples . UNESCO/CODE 
(Paris, 1986) p. 14. 



(3) ICAE, The World of Literacy : Policy, research and action . 

International Council for Adult Education (Ottawa, 1979) p. 8. 
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The introduction of techniques for remedial education at the 
L.E- in Lebanon and at the initial stages of preparatory schools in Syria 
in order to reduce drop-out and attrition rates are expected to be 
developed during 1992- The measures proposed to the FEDO's and the DUA's 
of Lebanon and Syria are delineated more extensively in separated field 
visit reports. The needed resources for the initation of those pilot 
experiences are assessed in these reports (4), 



Developing Diagnostic Testing : Present and Future Developments in $ED > 

5. In order to evaluate quality of education in UNRWA schools SED 

is setting up an important priority: the preparation and development of 
reliable tools for assessment and evaluation. This aspect of the 
strategy calls, as suggested by UNESCO, for diversified research : 

What is "quality" ? Current debate points mostly to the 
cognitive domain, measured in terms of knowledge and 
intellectual abilities such as those of analysis and 
synthesis. This is not surprising, since measurable criteria 
are, at least to some extent, more reliable than 
interpretations of observation. On the other hand, it is felt 
that "quality in education" cannot be totally restricted to 
the cognitive level. One has to cross frontiers within homo 
sapiens and examine the wide range of non-cognitive areas, 
bearing such names as "affective development", "social 
learning" and "moral education" (5). 



(4) CSED, Field Visit Report * Drop-out and efficiency in preparatory 
schools in the UNRWA / SAR Field\ Dept. of Education / SED 
29-30 December, 1991 • 

Field Visit Report to Lebanon, UNRWA Department of Education 
18-20 December, 1991 • 



(5) UNESCO, * Education for All • International Yearbook of Education. 
Vol- 36 (1984). p 38- 



r ^ 



8 



I 
I 
I 



- 6 - 



6. Following the above SED Specialists are at present working in 

the development of diagnostic instruments along three basic areas : 

(a) basic literacy and numeracy skills in the elementary level 
(Cognitive level); 

(b) psychomotor skills and physical fitness of elementary school 
children; and 

(c) impact of school children victims of organized violence in Lebanon, 
Gaza and West Bank. 

This diversified strategy will help to assess the quality of 
education in the first stages of primary education in the first 
instance, but later on, it is expected that these instruments could be 
developed to assess the upper grades of primary and preparatory levels. 
It is expected also that apart from the purely cognitive capacities, 
the psychomotor abilities as well as the affective aspects concerning 
school children will be diagnozed. 



Preparation and implementation of diagnostic instruments fo r assessing 
basic numeracy and literacy skills in SED 

7. The goal of the diagnostic test being prepared at SED for UNRWA 

schools is to identify the specific problems a student or a group of 
students is having vis-a-vis the numeracy and literacy skills that 
according to their level of age and maturity, they are suppose to 
acquire and master. 



A diagnostic test evaluates a particular sub-skill in much 
greater detail than is possible for an achievement test, which must 
cover many general areas broadly. Diagnostic tests are not limited to 
identifying weakness in pre-requi site skills and abilities. They can 
also be used for diagnosis well into the school year to try to determine 

the cause of repeated learning difficulties "Since a diagnostic 

test is designed to categorize weaknesses among students who are below 
average on a sub-skill, those instruments have a greater proportion of 
easy items than do achievement tests, which are designed measure the 
entire range of attainment" (6). 



Along those basic methodological guidelines a team constituded 
by the Special Education, Mathematic and Language Specialists from 
SED, liave been taking stock of existent experiences in Gaza, Jordan and 
the other Fields of Operations in order to produce a first draft 
diagnostic test. 



(6) Bloom, B et al .. Evaluation to Improve Learning (New York, 1981) 

p. 140. 
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It is expected that this first instrument will be 
experimentally applied in Burj Al Barajneh Camp in Lebanon by mid-March. 
The results will be analized and will have the double purpose of 
improving the reliability of the test as well as to diagnose the 
weaknesses in learning of literacy and numeracy of children in the fourth 
grade of primary education. 



Also on the basis of this results children in need of 
corrective work will be referred to a special remedial class to be 
Initiated as a Pilot experience in the same area - (Yarmouk School) 



Structure of the instrument and problems encountered (Fig 1) 

8. Three basic components of the diagnostic instrument are being 

developed by the team. 

(a) A detailed "map" of skills and sub--skills sequenced according to 
the age and/or materity of the child in the area of mathematics and 
language covering the contents from the first to the third level 
primary education* This instrument has been build on the basis of 
the existent curricula in the five fields of operation and also as 
the result of the specialists previous experiences in the analysis 
of achievement tests in their specializations* 

(b) On the basis of the "map" , items for testing have been written and 
developed by specialists, taking stock from existent standarized 
diagnostic tests and from achievement tests prepared in different 
fields, especially Gaza and Jordan. 



On this aspect, validation of each item is being done on the basis 
of exchange of expertise from local universities, UNICEF West Bank 
and UNRWA supervisors. 

(c) Following the experience of UNRWA Gaza, a follow-up instrument for 
individual and group assessment is in the process of being adapted 
and prepared. 

This instrument is most important given that it serves as a basic 
tool for identifying individual and group weaknesses in the process 
of literacy and numeracy skills learning. Thus, on the basis of 
this instrument it is possible to : 
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(i) Provide individualized remedial instruction to under- 
achievers. 

(11) Identify global weaknesses in the teaching-learning 

process with respect to specific sub-skills. By doing so 
it is then possible to reinforce via teacher training and 
teaching resource development the enrichment of educational 
activities in a rather specific and detailed basis for the 
UNRWA educational system at field level. In this way the 
instrument serves the purpose of an "early warning system". 

The development of a standarized diagnostic test is a collective and team 
effort 

9. Finally^ it should be said that, developing a diagnostic 

instrument for evaluating literacy and numeracy in UNRHA schools 1s 
confronted to the lack of expertise and experience in the development of 
these type of instruments in Arabic Language. 



The process of validity and reliability of a test is linked to 
expert consultation and adequate pre-testing and sample analysis in order 
to produce a test that will become t^table and consistent when 
administeired in the different fields of operation of UNRWA. 



External consultancy and the expert opinion of experienced 
scholars and practitioners from the Middle East and/or the international 
level seems most appropriated. Specially, if one considers that this 
experience is unique in the arab speaking world and will benefit not only 
UNRHA children but Palestinian and arab speaking children in general. 



10. Moreover, an open criteria of collaboration with other UN 

sisters agencies like UNESCO and UNICEF, as well as international and 
lornl NGO's (i.e Rada Barnen from Sweden ; Tamer Institute West -Bank) 
or universities , with a specific interest on the development or 
instruments for assessing the quality of instruction, should be consider 
as a most valuable contribution to this goal. This process of academic 
and professional exchange will also benefit th? specialist team of SED 
and will help to reinforce team and interdisciplinary work. 

The proposal submitted by SED to Stichteling Vluchteling and 
accepted by this NGO in order to provide and reinforce expertise on 
diagnostic testing and remedial education is a good example of this type 
of collaboration. 
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